Generating Exponentially Smaller POMDP Models Using Conditionally Irrelevant Variable Abstraction
نویسندگان
چکیده
The state of a POMDP can often be factored into a tuple of n state variables. The corresponding flat model, with size exponential in n, may be intractably large. We present a novel method called conditionally irrelevant variable abstraction (CIVA) for losslessly compressing the factored model, which is then expanded into an exponentially smaller flat model in a representation compatible with many existing POMDP solvers. We applied CIVA to previously intractable problems from a robotic exploration domain. We were able to abstract, expand, and approximately solve POMDPs that had up to 10 states in the uncompressed flat representation.
منابع مشابه
Planning with Abstraction Hierarchies Can Be Exponentially Less Eecient
It is well-known that state abstraction can speed up planning exponentially, under ideal conditions. We add to the knowledge|showing that state abstraction may likewise slow down planning exponentially, and even result in generating an exponentially longer solution than necessary. This phenomenon can occur for abstraction hierarchies which are generated automatically by the Alpine and Highpoint...
متن کاملRegion-Based Dynamic Programming for POMDPs
We present a major improvement to the dynamic programming (DP) algorithm for solving partially observable Markov decision processes (POMDPs). Our technique first targets the cross-sum pruning step of the DP update, a key source of complexity in POMDP algorithms. Unlike previous approaches, which reason about the whole belief space, the algorithms we present divide the belief space into smaller ...
متن کاملApproximate Planning in POMDPs with Macro-Actions
Recent research has demonstrated that useful POMDP solutions do not require consideration of the entire belief space. We extend this idea with the notion of temporal abstraction. We present and explore a new reinforcement learning algorithm over grid-points in belief space, which uses macro-actions and Monte Carlo updates of the Q-values. We apply the algorithm to a large scale robot navigation...
متن کاملCasson Fluid Flow with Variable Viscosity and Thermal Conductivity along Exponentially Stretching Sheet Embedded in a Thermally Stratified Medium with Exponentially Heat Generation
The motion of temperature dependent viscosity and thermal conductivity of steady incompressible laminar free convective (MHD) non-Newtonian Casson fluid flow over an exponentially stretching surface embedded in a thermally stratified medium are investigated. It is assumed that natural convection is induced by buoyancy and exponentially decaying internal heat generation across the space. The dim...
متن کاملMarkovian State and Action Abstractions for MDPs via Hierarchical MCTS
State abstraction is an important technique for scaling MDP algorithms. As is well known, however, it introduces difficulties due to the non-Markovian nature of state-abstracted models. Whereas prior approaches rely upon ad hoc fixes for this issue, we propose instead to view the state-abstracted model as a POMDP and show that we can thereby take advantage of state abstraction without sacrifici...
متن کامل